Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes for sequential I/O issues #11

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

gregs1104
Copy link

First round of bug fixes for new issue #10 and its related documentation.

I also made a small change renaming "exact" to "single" in the parameters. I think the documentation and even the code is easier to follow like this. I isolated that to its own commit, so it's possible to revert just that if you liked the original name better.

The original text only mentioned random reads.
os.lseek needs the file descriptor number of the file handle
Cleanup some obsolete function text too.

This is just good general paranoia to keep results on short runs
from being inflated by failed reads.  There's no path out of the
code yet that might count a read without doing one.
A few code assumptions didn't hold for blocksize < 512.
This avoids the potential exploits of negative sizes too.
Relying on a read to return no bytes was a fragile way to do wrap-around.

Sequential read wrap-around was depending the EOF behavior of read() to
return 0 bytes.  That isn't the most robust approach possible.  As one
example, in cases cases where files/devices were not an exact multiple
of block_size, this was counting the leftover small reads at the end
of the file/device as a full read.  That could inflate results a good
bit at the end of runs where the block size became very large.

Counting blocks also allows the seq read logic to be refactored to
start at any position.  The way all the sequential read threads were
synchronized to start at the same spot is itself a problem.  This
fix was needed first.

New code was tested with some temporary logging and a test long enough
here to wrap around to the start:

dd if=/dev/zero of=test.bin bs=1M count=1000
./iops -n 1 -b 512 -p sequential -t 20 test.bin

1468848531.4 count=2047997 seq to pos 2047997 out of 2048000 blocks
1468848531.4 count=2047998 seq to pos 2047998 out of 2048000 blocks
1468848531.4 count=2047999 seq to pos 2047999 out of 2048000 blocks
1468848531.4 count=2048000 seq to pos 0 out of 2048000 blocks
1468848531.4 count=2048001 seq to pos 1 out of 2048000 blocks
1468848531.4 count=2048002 seq to pos 2 out of 2048000 blocks

Here's the temporary debug lines:

          if pattern=='random':
              pos = random.randint(0, mediasize - blocksize) # need at least one block left
              pos &= ~(sectorsize-1)   # sector alignment at blocksize
              fh.seek(pos)
+         else:
+             print time.time(), "count=%d seq to pos %d out of %d blocks" % (count,seq_block,blocks)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant